\chapter{Conclusion}

This dissertation shows that, by utilizing the idea of separating
data from metadata, a storage system can achieve strong robustness
guarantees with little impact on efficiency and scalability. This
dissertation demonstrates the power of this idea by applying it to
three very different systems.

I have learned several lessons during my work. First, the general
belief that stronger protection is more expensive, which is often
proved to be true in a theorem, may not be an insuperable
obstacle when coming to real systems: real systems
are often complex and may deviate from the general model to which the theorem
applies in subtle ways, opening up new opportunities for optimizations. Finding
such opportunities usually requires a deep understanding of the theorem
itself, and in particular its assumptions.

Second, when calculating the cost of a protection technique, what really
matters is its cost in the failure-free case, because this is the most
common scenario: optimizing cost in the failure-free case is the key
in both Gnothi and Salus, and their costs in the presence of failures are actually
not different from those of previous works. Although this is not a new
insight, two things are worth noting: first, the replication thresholds proved in
different theorems (e.g. $2f+1$ for asynchronous replication) often apply
to worst-case scenarios, which might be misleading to some extent. Second,
optimizing the failure-free case should not significantly hurt the availability
of the system when failures occur: ensuring this property is exactly the
key factor that distinguishes Gnothi from previous works.


Finally, although the size of metadata is small, how to process metadata
still requires careful thoughts. For example, caching all metadata in memory
can cause memory overhead while storing it to disks may cause random disk accesses:
neither option is desirable. My experience in working on this dissertation
suggests that identifying an effective way to reduce the overhead of processing
metadata usually requires a significant engineering effort.

In conclusion, strong protection of data does not have to be expensive.
Many systems that are making a tradeoff between robustness and
scalability can actually enjoy the benefits of both,
as long as we can design and protect metadata properly. 